The technology disclosed includes a software architecture for methods and devices used to test routers, switches and the like, particularly high volume infrastructure devices. In particular, this architecture associates test data stream definitions with emulated devices that send test frames or packets to and receives them from a system or device under test (“DUT”). By relating the test data stream definitions to definitions of the emulated devices that are coupled to the DUT, automation of test frame and/or test packet design improves, especially where some of the information needed to generate test frames and/or packets is dynamically generated during a test.
Testing of network devices with a single test device is difficult because of the variety of test types, devices to test, interfaces to the devices, and control plane protocols that they run. These factors combine into test scenarios. Tests can be directed to conformance testing, functional testing, performance testing or passive testing, among others. Sample devices include routers, switches and hybrid devices. Typically, these devices are at the edge or core of a communications infrastructure such as the Internet or a carrier network. Interfaces to network devices include RJ-45 copper, fiber (single mode, multi-mode, SFP GBIC, and SFP MSA capabilities), XENPAK MSA, XFP MSA, X2 MSA and 10GBASECX4. Control plane protocols include BGP, DHCE, DHCP-PD, GRE, IGMP/MLD, L2TP, MEF9, MPLS/LDP/RSVP-TE, multicast routing, OSPF, PPPoX, RIP, STP/MSTP/RSTP, 802.1ah and 802.3ad. Data plane protocols include HTTP, FTP, Telnet, DNS, SSL, SMTP/POP3, IPSEC, IPv6, IPTV, SIP, and MMS.
Standards have been set for testing devices. Some of these standards are promulgated by ITEF following a request for comments procedure and are referred to by RFC number. Some of the test measurement standards include RFC-2544 and RFC-2889. Scenarios tested include VLAN Network Device Benchmarks, (between enterprise networks and with an enterprise backbone network), IPTV testing, Layer 2/3 switching and routing between enterprise networks and service provider networks, Spanning Tree Protocol behavior and performance upon failure of a route leg, multicasting expansion and propagation, Border Gateway Protocol router testing.
FIG. 1 depicts a sample test topology for layer 2-3 switching and routing between enterprise networks and service provider networks. The device under test sits between the networks. In this figure, the emulated enterprise network includes virtual LAN or “VLAN” segments. Traffic on the enterprise network includes voice, data and video. On the emulated service provider in our side of the DUT, sources of voice, data and video content are represented. The switching and routing devices in this scenario may support a large number of layer 2/3 technologies, including different virtual LAN encapsulations, various quality of service (“QOS”) schemes, and dual-stack IPv6/IPv4 forwarding. Testing is particularly useful because each new feature or protocol increases the risk of functional or performance problems, many of which are hard to find without creating and analyzing a wide range of traffic scenarios.
The testing industry standard architecture for authoring test packets has involved authoring network sessions and raw test data packets, following a particular order that recognizes the dependencies between sessions that set up routes and establish addressing and the protocols that operate within the session context. For instance, in the TestCenter™ product v1.4 from Spirent Communications, a test engineer who was testing MPLS traffic would (1) define physical and logical characteristics of a port; (2) define upstream and downstream LDP ports and LSP sessions; (3) copy parameters, such as Forwarding Equivalence Class values and prefixes, from the upstream and downstream sessions into raw test data packets to be transmitted in the context of these sessions. To help organize the variety of port, protocol and packet choices to be made, authoring wizards have been provided at the scenario level. This approach is described more fully in Reference Manual, Spirent TestCenter™ System (August 2006), 134 pp., which is hereby incorporated by reference to illustrate the prior art approach and the variety of test scenarios addressed. The test set-up approach described in the Reference Manual has become familiar to and accepted by test engineers.
FIG. 2 depicts a tree of configuration options, including control plane protocols, implemented in the test center product version 1.4. Hardware components that present this tree of configuration options included high density network test chassis designated as SPT-9000A, SPT-5000A and SPT-2000A. A variety of test modules or cards can be mounted in these chassis. The first set of configuration options in FIG. 2 is for setup of cards, including setting up ports on cards and configuring ATM, VLAN and IP options on the ports. An alphabet soup list of control plane protocols is presented, including STP, BGP, OSPF, OSPFv3, RIP, IS-IS, RSVP-TE, LDP, I GMP and PIM. These control plane protocols each correspond to a network control session. The control plane protocols are presented for configuration of corresponding sessions. A user would expand a protocol choice, such as LDP, and configure one or more corresponding network control plane sessions. After configuring the control plane sessions, a user would expand the traffic tab, configure ports on the card as transmit and receive ports, and configure test frames or test packets in raw streams. The complexity of configuring test that include both control plane sessions and data plane raw streams can be imagined from the presentation of so many choices as individual special cases. A very simple approach to data structures has been used to represent network connections and data streams.
FIG. 3 depicts a network session object 311, a raw test packet object 313, 315 and corresponding test implementation objects 321, 323. The application configuration information 310 was entered on a PC or laptop that handled so called business layer logic. This configuration information was communicated to port 320 running on a firmware test module. The firmware test module was in the chassis, mentioned above, and include both general purpose computing logic and field programmable gate arrays (FPGAs) for high speed packet generation. In the software design structure depicted, a network session object and a raw test packet object represent distinct parts of the test. The details of the network session data structure were highly dependent on the particular network protocol involved. Network protocols were treated as special cases and handled by specialized code. While an LDP session 311 is depicted as communicating with a generalized routing daemon 321, in fact, specialized routing daemons were implemented on the test module for each type of network session. Raw test packet objects were constructed after network sessions, as address and label fields of the raw test packets were copied from the network session objects into the test frames. Test engineers developed significant skills while learning which raw test packet fields depended on which network session fields for the many special cases of network protocols. Some wizards were provided to assist with initial set-up consistency between network session and test packets set up, but test engineers were on their own for maintaining consistency once they began editing data fields generated using the wizard. A raw test packet 313 might, for instance, including Ethernet, MPLS and IP address information for both source and destination. The raw test packet information 313 would be provided to a generator 323 on a port 320 for packet generation. The data structures in this model correspond very literally with the network
From the enormously complex array of choices presented, an opportunity has been realized to change the representation of components and the model for test configuration. A much different underlying representation may improve and simplify data entry, initial generation of prototype packets, dynamic discovery of address information, and updating address information during a test. More reliable and economical systems may result.